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DETAILED ACTION 

1 . The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 

Continued Examination Under 37 CFR 1.114 

2. A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. 

Information Disclosure Statement 

3. The references listed in the Information Disclosure Statement submitted on 05/27/2008 
have been considered by the examiner (see attached PTO-1449). 



Response to Amendment 

4. This communication is responsive to the applicant's amendment and RCE both filed on 
05/27/2008. The applicants ) amended claims 1, 15 and 29 (see the amendment: pages 2-8). 
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Response to Arguments 

5. Applicant's arguments filed on 05/27/2008 with respect to the claim rejection under 35 
USC 103, have been fully considered but are moot in view of the new ground(s) of rejection, 
since the amended claims introduce new issue and/or change the scope (ground) of the claims. 

It is noted that the previous cited references are still applicable to the newly amended 
claims for the prior art rejection with new ground, because the combined teachings of the 
references can be properly read the amended claims (see detail below). 

Claim Rejections - 35 USC §103 

6. Claims 1, 15 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
ALBAYRAK et al. (US 6,662,163 Bl) hereinafter referenced as ALBAYRAK, in view of 
WHITE et al. (US 6,460,017 Bl) hereinafter referenced as WHITE. 

As per claim 1, ALBAYRAK discloses system and method for programming portable 
devices from a remote computer system (title), in client/server environment (Fig. 1), comprising: 

"a set of controls configured for use on server remote from the client for defining a dialog 
in an authoring page for a website and used to dynamically generate client side markup in 
accordance with the dialog", (col. 3, lines 51-55, 'dynamically program portable client 
computer', 'manage voice dialogs for the purpose of interacting with and guiding users in 
various work tasks (necessarily including the corresponding various controls)'; col. 4, lines 24- 
28, 'voice browse read a VoiceXML (markup) page. . .and acts upon the information and 
instructions (a set of controls); col. 7, line 31 to col. 8, line 67, 'web server (i.e. website)' 
includes 'VoiceXML pages 250. . .are generated (creating web pages — i.e. authoring page) by the 
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SHIM 242 (on server —website)', 'VoiceXML templates 252' also necessarily/inherently 
includes a set of controls because 'each of which describes an application- specific verbal dialog 
(defining a dialog)', 'system configuration data' including 'the VoiceXML page currently loaded 
in the client (corresponding to dynamically generate client side markup)' and 'grammar files 
270, which are used by the VoiceXML browser in the clients to map words spoken (i.e. using a 
grammar for recognition) by a user to a legal response expected by the application that is 
controlling (including controls) the operation of the system' (also see col. 10, lines 25-44); Fig. 
5A-B, wherein combined 'CGI program' and 'application socket SHIM' can also correspond to 
the claimed a set of controls), 

"the controls comprising at least a question control for generating markup related 
to audible prompting of a question, and an answer control for generating markup related 
to a grammar for recognition", (col. 4, lines 22-67 and 'VoiceXML . . .to create audio 
dialogs', 'voice browse read a VoiceXML (markup) page. . .and acts upon the information 
and instructions (controls)', 'user's voice files and application-specific grammar files 
(both need for speech recognition) are loaded on the client', 'play an audio prompt 
(interpreted as question control)' and 'wait for user to confirm (interpreted as answer 
control) that he completed the requested action'; Fig. 2 and col. 6, lines 25-40, 
'affirmative or negative response including "yes", "no" (corresponding to answer 
control)' that which further suggests that the system provides a "question" related control 
before this response and the "answer" control for handling this response, wherein the 
'voice prompt' is provided by 'using prerecorded digital speech' or 'text-to-speech 
synthesis'), 
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"each of the controls having attributes [including an attribute to indicate whether 
the associated control is available for activation], wherein the control are arranged in a 
hierarchical structure and repeatedly used with attributes having different values in order 
to define the dialog" (col. 8, lines 1-28, 'VoiceXML pages. . .are generated from one of a 
set of VoiceXML templates', 'each of which describes an application- specific verbal 
dialog'; col. 9, lines 13-60, 'for a particular situation and task (control), the SHIM 
provides values for the macro (control) variables (attribute)', 'different templates provide 
dialog structures for different application (also read on controls) such as picking, returns 
processing, and inspection', and the example of a VoiceXML template shows the 
attributes (such as application, name, expr) are assigned (directly related to) values (such 
location, &wms_location) and has a hierarchical structure and the associate 
corresponding grammars (also see col. 10, lines 25-44 and Fig.4 and Fig. 5 A); col. 12, 
lines 3-37, discloses using VoiceXML page for training steps that 'are repeated until the 
voice browser terminates training', wherein one of ordinary skill in the art would have 
readily recognized that the templates and the corresponding control(s) with the attribute 
(such as name, expr in the example) would be necessarily or inherently repeatedly used); 
"a module, using the attributes provided directly from the controls in the authoring page 

and when executed on the server, generates client side markup based on the dialog [as a function 

of which controls are activated]" (Fig.4 and col. 8, lines 1-8, 'system host interface module 242'; 

col. 8, lines 1-28 and col. 9, lines 12-60, as stated above; also see col. 11, lines 21-29); and 

"a module configured for use on the client and when executed on the client and using the 

client side markup, creates a dialog [as a function of which controls are activated]", (Fig. 2 and 
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col. 3, line 6-64, 'the voice browser (module used on the client) interprets voice pages received 
from the server. . .then performs (executed on client) an action based on the text response', 
'dynamically program portable client computers, and to manage voice dialogs for the purpose of 
interacting with and guiding users in various work tasks'; col. 4, lines 24-2, 'the voices browser 
reads a VoiceXML (markup) page', and 'VoiceXML was designed ... to create (generate) audio 
dialogs that feature digitized audio and speech recognition'). 

ALBAYRAK does not expressly disclose "each of the controls having an attribute to 
indicate whether the associated control is available for activation" and generating/creating the 
dialog "as a function of which controls arc activated". However, the feature is well known in the 
art as evidenced by WHITE who, in the same field of endeavor, discloses distributed voice web 
architecture and associated components and methods (title), and teaches that 'an attribute of a 
component is information the component has' (col. 9, lines 20-21), and provides 'endpointer' 
with attribute (or parameter): 'active: boolean' for reporting (indicating) the outcome of user's 
utterance activation according to the related dialog steps or events (associated controls available) 
and 'firing (generating/creating)' dialog related events accordingly (col. 20, lines 14-64), which 
is broadly interpreted as generating/creating a dialog as "function of which controls are 
activated". Therefore, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify ALBAYRAK by providing attribute reporting the outcome of 
activation for dialog steps or events, as taught by WHITE, for the purpose (motivation) of 
generating the prompt in response to the recognized speech and/or transmitting the prompt to the 
remote device (WHITE: col. 1, lines 50-53). 
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In addition, in another view, ALBAYRAK discloses the dialog between 'client' and 
'server' and 'voice browser interpret: page play audio prompts (question control) and wait for 
user's verbal response(s) (answer control)' (Figs. 5A-5B), and teaches that 'voice 
browser. . .follows its instructions (set of controls) to carry on the application- specific dialog with 
user (334)' (col. 11, lines 31-32), one of ordinary skill in the art would readily recognized that 
this system necessarily includes activation information (attribute) indicating current running 
tasks (controls) and operative function in response to an interactive action (activating the 
function, such as playing prompts or waiting an response), in order to keep normal operation for 
the dialog interactions; and 'manage voice dialogs' for 'interacting with and guiding users in 
various work tasks' between 'client and server computers' and 'control how the client interact 
with the user', and 'the server "servers" VoiceXML pages to voice browsers on portable client' 
using the WWW protocols (or HTTP)' (col. 3, line 51 to col. 4, linelO), which necessarily and/or 
inherently includes various elements of ML and/or VoiceXML language, including forms, 
controls, function calls (sub-dialog) for invoking (activate) and interacting the dialog (see detail 
inherent evidence in IDS: document No. CA "Voice XML Forum" filed 12/29/2006). Therefore, 
it would have been obvious to one of ordinary skill in the art at the time the invention was made 
to recognize that the dialog (form and menu) and sub-dialog (function call) defined in 
VoiceXML are preferably used for invoking (activating) and interacting voice dialog, and to 
modify ALBAYRAK by providing activation information (attribute) indicating current running 
tasks (activated controls) and operative function in response to an interactive action in the dialog 
by using VoiceXML, for the purpose (motivation) of performing an action based on text 
response converted by voice browser (ALBAYRAK: col. 3, lines 11-13). 
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As per claim 15, the rejection is based on the same reason described for claim 1, because 
it also reads on the limitation(s) of claim 15, wherein teachings of 'the VoiceXML page 
currently loaded in the client' and 'grammar files 270'used by 'the VoiceXML browser in the 
clients to map words spoken (i.e. using a grammar for recognition) by a user to a legal response 
expected by the application that is controlling (including controls) the operation of the system' 
(col. 8, lines 9-23; also see col. 10, lines 25-44); 'different templates provide dialog structures for 
different application (also read on controls)' and the 'example of a VoiceXML template 
(control)' having the attributes and their values associated with a recognition application and/or 
location of grammar (file) for the application (col. 9, line 12 to col. 10, line 44), 'voice prompt' 
provided by 'using prerecorded digital speech' or 'text-to-speech synthesis' (col.6, lines 25-40), 
from ALBAYRAK, are read on the claimed "an attribute directly related to using a grammar for 
recognition", "location of grammar for use in recognition", and "at least one of inline text for 
text-to-speech conversion, location of data for audible rendering and playing of a prerecorded 
audio file" respectively. 

As per claim 29, it recites a method. The rejection is based on the same reasons described 
for claim 15, because the method claims and apparatus claims are related as apparatus and 
method of using same, with each claimed element's function corresponding to the claimed 
method step. 



7. Claims 2-5, 7-9, 11, 14, 16-19, 21-23, 25, 28 and 30-34 are rejected under 35 U.S.C. 
103(a) as being unpatentable over ALBAYRAK in view of WHITE as applied to claim 1, and 
further in view of ALPDEMIR (US 2002/0035474 Al). 
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As per claim 2 (depending on claim 1), ALBAYRAK in view of WHITE does not 
expressly disclose "the question control activates the answer control". However, the feature is 
well known in the art as evidenced by ALPDEMIR who, in the same field of endeavor, discloses 
voice-interactive marketplace providing time and money saving benefits and real-time promotion 
publishing and feedback (title), comprising 'natural language recognition' that 'listens users' 
request in free form speech or extracts the command and/or data, . . .asks additional questions of 
the users '(paragraph 220) and the interactions in the dialog including question/answer sequences 
(paragraphs 253-268), which suggests the system has capability of implementing the claimed 
"question control activates the answer control". Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify ALBAYRAK in view of 
WHITE by specifically providing question/answer interactions in the dialog, as taught by 
ALPDEMIR, for the purpose of providing greater interactive capability and/or communicating 
the speech-based representation of the particular data item to the external device (ALPDEMIR: 
col. 3, lines 1-2 and abstract). 

As per claim 3 (depending on claim 2), ALBAYRAK in view of WHITE and 
ALPDEMIR further discloses "a command control for generating markup related to a grammar 
for one of navigation in the markup, help with a task, and repeating an audible prompt", 
(ALBAYRAK: col. 11, lines 31-32, 'voice browser. . . follows it instructions (command control) 
to carry on the application-specific dialog with user (334)'; col. 4, lines 23-24, 'voice 
XML(markup). . .to create (generate) audio dialogs that feature digitized audio and speech 
recognition'; col. 4, lines 23-24, 'repeating the telling (audio prompt-telling)'; ALPDEMIR: 
paragraphs 132 and 140, ' "help" there should desirably be some automated help (task)' and 
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'standard data and information formats and protocols, such as HTML, XML, and XVML 
(markup)'). 

As per claim 4 (depending on claim 4), ALBAYRAK in view of WHITE and 
ALPDEMIR further discloses "a confirmation control for generating markup related to 
confirming that a recognized result is correct", (ALBAYRAK: col. 4, lines 63-66). 

As per claim 5 (depending on claim 4), ALBAYRAK in view of WHITE and 
ALPDEMIR further discloses "the confirmation control is activated as a function of a confidence 
level or a received result", (ALPDEMIR: paragraph 176, function of 'confirm information 
{confirm()}'). 

As per claim 7 (depending on claim 5), ALBAYRAK in view of WHITE and 
ALPDEMIR further discloses "the confirmation control activates an accept control to accept the 
recognized result", (ALPDEMIR: paragraph 191, sub-grammar "yes"). 

As per claim 8 (depending on claim 5), ALBAYRAK in view of WHITE and 
ALPDEMIR further discloses "the confirmation control activates a deny control to deny the 
recognized result", (ALPDEMIR: paragraph 191, sub-grammar "no"). 

As per claim 9 (depending on claim 5), the rejection is based on the same reason 
described for claim 4, because the claim recites the same or similar limitation(s) as claim 4. 

As per claim 11 (depending on claim 2), ALBAYRAK in view of WHITE and 
ALPDEMIR further discloses "the answer control includes a mechanism to associate a received 
result with one control of the set of controls", (ALPDEMIR: paragraph 191, sub-grammar "yes" 
or "no"). 
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As per claim 14 (depending on claim 1), ALBAYRAK in view of WHITE further 
discloses "a second set of controls for generating markup related to visual rendering on a client", 
(ALBAYRAK: col. 2, lines 19-20 and 46 and 45-46 'displaying . . .information from XML files' 
and 'voice browser that interprets VoiceXML programs similar to the way computer users use a 
graphical browser (including second set of controls) that interprets HTML program'), but 
ALBAYRAK in view of WHITE does not expressly disclose "wherein each control of the first- 
mentioned set of controls is associated with at least one of the controls of the second set of 
controls". However, the feature is well known in the art as evidenced by ALPDEMIR who 
further discloses that 'a message is played/displayed when a caller request businesses. . .the right 
to be announced/displaycd'(paragraph 85), 'the text information from the data is converted to 
speech. . . and played backed to the caller using the caller's devices 106' and 'the information 
database 134. . .can also be accessed with a display device' including wireless phones, PDA or 
palmtop . . .with the ability to display standard HTML', and suggest using 'standard data and 
information formats and protocols, such as HTML, XML, VXML' (paragraphs 138-139), which 
suggests the system has capability of associating voice related action with text related action by 
using VXML, HTML or XML for playing and displaying. Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
ALBAYRAK in view of WHITE by specifically providing functionality of playing voice and 
displaying text (or graphics) by using VXML, HTML or XML, for the purpose of providing 
greater interactive capability and/or communicating the speech-based representation of the 
particular data item to the external device (ALPDEMIR: col. 3, lines 1-2 and abstract). 
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As per claims 16-19, 21-23, 25 and 28 (depending on claim 15), the rejection is based on 
the same reason described for claims 2-5, 7-9, 1 1 and 14 respectively, because the claims recite 
the same or similar limitation(s) as claims 2-5, 7-9, 1 1 and 14 respectively. 

As per claims 30-34 (depending on claim 29), the rejection is based on the same reason 
described for claims 2-5 and 14 respectively, because the claims recite the same or similar 
limitation(s) as claims 2-5 and 14 respectively. 

8. Claims 6, 12-13, 20 and 27 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over ALBAYRAK in view of WHITE and ALPDEMIR, as applied to claims 5, 1 1, 19, 25, and 
further in view of CHINN et al. (US 2002/0010715 Al) hereinafter referenced as CHINN. 

As per claim 6 (depending on claim 1), even though ALBAYRAK in view of WHITE 
and ALPDEMIR discloses finding match of an answer (ALPDEMIR: paragraph 228) and using 
'check score' function (ALPDEMIR: paragraph 176), ALBAYRAK in view of WHITE and 
ALPDEMIR does not expressly disclose "the answer control includes an attribute related to a 
confidence level". However, the feature is well known in the art as evidenced by CHINN who, 
in the same field of endeavor, discloses system and method of browsing using a limited display 
device (title), providing accessing web content by using voice commands and markup language 
(paragraph 6), and teaches that 'the confidence score is a value used by the system that 
represents the level of certainty in recognition' and "the system may reject a request if the 
confidence score is below a specific threshold, or may attempt to determine with more certainty 
(i.e., disambiguate) a request with a confidence score that falls within a specific range' 
(paragraphs 186 and 222-224). Therefore, it would have been obvious to one of ordinary skill in 
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the art at the time the invention was made to modify ALBAYRAK in view of WHITE and 
ALPDEMIR by specifically providing information (attribute) for finding a match by using 
confidence score, as taught by CHINN, for the purpose of representing the level of certainty in 
recognition (CHINN: paragraph 186). 

As per claim 12 (depending on claim 1 1), the rejection is based on the reason described 
for claim 6, because it also reads on the limitations of claim 12. 

As per claim 13 (depending on claim 12), ALBAYRAK in view of WHITE, ALPDEMIR 
and CHINN further discloses that "the mechanism includes issuing an event related to operation 
of binding", (WHITE: col. 10, lines 41-47, 'an cvcnt-drivcn architecture' and teaches that 
'events are "fired" (signalcd)(read on issued) by the browser, . . .or a content application'). 

As per claims 20 and 27 (depending on claim 15), the rejection is based on the same 
reason described for claims 6 and 13 respectively, because the claims recite the same or similar 
limitation(s) as claims 6 and 13 respectively. 



Conclusion 

9. Please address mail to be delivered by the United States Postal Service (USPS) as 
follows: 

Mail Stop 

Commissioner for Patents 

P.O. Box 1450 

Alexandria, VA 223 13-1450 
or faxed to: 571-273-8300, (for formal communications intended for entry) 
Or: 571-273-8300, (for informal or draft communications, and please label 
"PROPOSED" or "DRAFT") 
If no Mail Stop is indicated below, the line beginning Mail Stop should be omitted from the 
address. 



Effective January 14, 2005, except correspondence for Maintenance Fee payments, 
Deposit Account Replenishments (see 1.25(c)(4)), and Licensing and Review (see 37 CFR 5.1(c) 
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and 5.2(c)), please address correspondence to be delivered by other delivery services (Federal 
Express (Fed Ex), UPS, DHL, Laser, Action, Purolater, etc.) as follows: 

U.S. Patent and Trademark Office 

Customer Window, Mail Stop 

Randolph Building 

Alexandria , VA 22314 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to QI HAN whose telephone number is (571)272-7604. The 
examiner can normally be reached on M-TH:9:00-17:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (571)-272-7602. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



QH/qh 

August 11,2008 
/Qi Han/ 

Examiner, Art Unit 2626 



